AIMS:  An  Agent-Based  Information  Management  System  in  JBI-like 

Environments 


Srikanth  Kallurkar,  Bin  Yu,  Jay  Askren, 
Lewis  Fishgold,  Ganesh  Vaidyanathan 
Donald  Steiner 

Quantum  Leap  Innovations,  Inc. 
Newark,  DE  19711,  USA 
{svk,  byu,  jda,  Ihf,  gv,  dds}@quantumleap.us 


Robert  Flo 

Air  Force  Research  Lcib/IFSB 
525  Brooks  Road 
Rome,  NY  13441 
Robert.  Flo@  rl.  af.  mil 


ABSTRACT 

One  of  the  challenges  for  many  mission-critical  applica¬ 
tions  is  how  to  determine  the  quality  of  information  from 
distributed  and  heterogeneous  data  sources.  In  this  paper 
we  consider  Joint  Battlespace  Infosphere  (JBI)  or  JBI-like 
collaborative  information  sharing  environments,  where  the 
quality  of  published  information  often  depends  on  the  qual¬ 
ity  of  the  contributing  information  and  of  the  sources  that 
publish  them.  However,  as  the  environment  becomes  larger 
and  more  diverse,  it  is  becoming  increasingly  difficult  for 
human  operators  to  assess  the  quality  of  information  from 
various  data  sources.  To  address  this  challenge,  we  develop 
AIMS,  an  Agent-based  Information  Management  System, 
to  manage  the  quality  of  information  in  JBI-like  environ¬ 
ments,  e.g.,  information  trustworthiness.  In  our  approach, 
each  operator  is  associated  with  a  software  agent,  called 
client  agent.  The  client  agent  enables  its  operator  to  inter¬ 
act  with  the  JBI  repository  via  the  sendees  of  query,  pub¬ 
lish,  and  subscribe.  Moreover,  the  client  agent  collaborates 
with  other  agents  to  assess  and  learn  the  trustworthiness 
of  information  and  the  reliabilities  of  corresponding  data 
sources  from  its  pedigree  and  the  feedback  from  operators. 

KEYWORDS:  Joint  Battlespace  Infosphere(JBI),  in¬ 
formation  sharing,  software  agents,  trust,  pedigree. 

1.  INTRODUCTION 

Collaborative  information  sharing  environments  such  as  JBI 
(Joint  Battlespace  Infosphere)  integrate  widely  dispersed 
human  decision  makers  with  various  data  sources  into  a 
highly  dynamic  information  sharing  system  [1],  The  JBI 
collects  data  from  a  wide  variety  of  sources,  aggregates 
this  information,  and  distributes  the  information  to  users 
at  all  force  levels  [9].  One  challenge  in  JBI-like  collabo¬ 
rative  information  sharing  environments  is  that  information 


provided  by  different  clients,  including  operators  and  data 
sources,  will  be  of  varying  quality,  while  many  mission- 
critical  applications  such  as  battlefield  operations  and  intel¬ 
ligence  analysis  often  require  trustworthy  and  high-quality 
information  [11].  Current  implementations  of  JBI  devel¬ 
oped  by  the  Air  Force  Research  Lab  and  other  agencies 
provide  a  rich  collection  of  functionalities  for  human  op¬ 
erators,  such  as  publish,  query,  subscribe,  but  none  of  them 
support  any  mechanisms  that  can  assess  the  changing  qual¬ 
ity  of  information  from  various  data  sources  in  dynamic  en¬ 
vironments  [8,  15,  16]. 

Assessment  of  information  quality  can  be  partially  captured 
by  its  trustworthiness  and  pedigree/provenance,  e.g.,  source 
and  history  [3],  For  example,  information  from  reliable 
sources  usually  has  higher  quality  than  the  one  provided  by 
unknown  or  questionable  sources.  However,  in  a  dynamic 
network  centric  environment  with  diverse  communities  of 
interests,  it  becomes  increasingly  difficult  for  human  oper¬ 
ators  to  record  and  analyze  the  history  and  trust  of  derived 
information  and  the  reliabilities  of  various  clients,  including 
both  data  sources  and  operators  [12]. 

In  this  paper  we  describe  AIMS,  an  agent-based  information 
management  system,  to  manage  the  quality  of  information 
in  JBI,  e.g.,  trustworthiness.  The  goal  of  this  work  is  to  de¬ 
velop  an  experimental  JBI  system  that  can  effectively  man¬ 
age  the  trustworthiness  of  information  in  a  highly  dynamic 
information  sharing  environment,  where  the  exchanged  in¬ 
formation  may  be  changed  and  merged  by  operators  after 
it  is  published.  We  demonstrate  the  use  of  agent  technol¬ 
ogy  in  AIMS  within  the  domain  of  the  intelligence  analysis 
community,  where  the  AIMS  system  automatically  extracts 
news  articles  from  a  number  of  RSS  (Really  Simple  Syndi¬ 
cation)  news  feeds  specified  by  its  user  and  stores  them  in  a 
repository. 

Specifically,  we  describe  how  pedigree  (or  provenance)  can 
be  used  to  support  and  enhance  the  assessment  of  the  trust¬ 
worthiness  of  information.  In  our  approach,  there  are  multi¬ 
ple  client  agents  (each  of  which  is  associated  with  an  oper- 
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including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 


1.  REPORT  DATE 

MAY  2007 


2.  REPORT  TYPE 


3.  DATES  COVERED 

00-00-2007  to  00-00-2007 


4.  TITLE  AND  SUBTITLE 

AIMS:  An  Agent-Based  Information  Management  System  in  JBI-like 
Environments 

6.  AUTHOR(S) 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

Air  Force  Research  Lab/IFSB,525  Brooks  Road, Rome, NY, 13441 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


5a.  CONTRACT  NUMBER 


5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 

11.  SPONSOR/MONITOR'S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 


13.  SUPPLEMENTARY  NOTES 

Proceedings  of  International  Symposium  on  Collaborative  Technologies  and  Systems  (CTS  2007),  21-25 
May,  Orlando,  FL 

14.  ABSTRACT 

see  report 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

unclassified 


b.  ABSTRACT 

unclassified 


c.  THIS  PAGE 

unclassified 


17.  LIMITATION  OF 

18.  NUMBER 

ABSTRACT 

OF  PAGES 

Same  as 

8 

Report  (SAR) 

19a.  NAME  OF 
RESPONSIBLE  PERSON 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


ator)  and  a  provenance  agent.  The  client  agent  collaborates 
with  the  provenance  agent  to  track  the  pedigree  of  the  infor¬ 
mation  and  learn  the  reliability  of  each  client,  e.g.,  an  ana¬ 
lyst,  a  news  source  or  a  sensor.  Moreover,  the  client  agent 
reasons  and  learns  about  the  trustworthiness  of  the  derived 
information  based  on  the  known  reliability  of  each  informa¬ 
tion  provider  in  the  pedigree  graph  and  their  trust  on  each 
object. 

The  rest  of  this  paper  is  organized  as  follows.  In  Section  2 
we  present  an  overview  of  JBI-like  information  sharing  en¬ 
vironments.  Section  3  describes  the  design  and  implemen¬ 
tation  of  AIMS  in  the  domain  of  intelligence  analysis.  We 
describe  our  approach  to  managing  the  trustworthiness  of 
information  in  Section  4.  Section  5  discusses  some  related 
work  in  tmst  management  and  Section  6  concludes  the  pa¬ 
per  with  suggested  directions  for  future  research  and  devel¬ 
opment. 

2.  JBI-LIKE  ENVIRONMENTS 

This  section  provides  some  background  on  JBI-like  infor¬ 
mation  sharing  environments,  such  as  information  objects, 
metadata,  clients,  and  information  pedigree. 

2.1  Information  Objects  and  Clients 

Joint  Battlefield  Infosphere  (JBI)  [1]  is  a  common  frame¬ 
work  for  tactical  information  sharing  and  dissemination 
among  commanders  in  different  levels  of  echelons  and  op¬ 
erators  in  the  battlefield  command  and  control  center.  The 
wealth  of  information  may  include  (1)  synthetic  aperture 
radar  (SAR)  imagery,  electro-optical/infrared  (EO/IR)  im¬ 
agery,  and  moving  target  indication  (MTI)  target  status  data; 
(2)  meteorological  and  oceanographic  data. 

In  JBI-like  environments,  information  is  stored  and  made 
available  in  the  form  of  information  objects.  The  JBI  plat¬ 
form  maintains  a  repository  of  such  information  objects. 
These  objects  are  statements  about  the  real  world,  such  as  a 
SAR  sensor  report,  a  recorded  UAV  video,  or  a  description 
of  available  fuel  stores  in  a  specific  area.  A  JBI  information 
object  contains  metadata  about  the  information,  information 
metadata,  as  well  as  the  information  itself,  information  pay- 
load  (see  Figure  1).  The  metadata  defines  a  common  set  of 
attributes  of  a  JBI  object,  such  as  version  number,  publica¬ 
tion  time,  publisher,  pedigree  (parents),  subject  keywords, 
language,  and  etc. 

Figure  2  describes  the  metadata  of  an  information  object 
in  the  domain  of  intelligence  analysis,  where  an  object  is 
a  news  article  from  a  given  news  source  such  as  Reuters, 
BBC,  VOA,  and  NPR.  From  the  metadata  we  can  find  the 
information  object  is  a  news  article  from  BBC  and  it  is  cre¬ 
ated  based  on  two  other  articles:  Reuters. 124325235  and 
NPRJ2352161.  The  metadata  also  includes  the  date  of  be¬ 
ing  created,  the  source  of  the  article,  the  URL  of  RSS  news 
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Figure  1:  The  information  metadata  and  payload  in  a  JBI 
information  object. 


feed  and  a  list  of  keywords  for  the  article. 


<?xml  version="  1 .0"  encoding=”UTF-8"?> 

<NewsArticle  xmlns="http://www.quantumleap.us/aims/news" 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> 

<DocumentIdentifier>BBC_l  16067508328  l</DocumentIdentifier> 

<Title>Gazans  fear  call  from  Israel</Title> 

<Source>BBC</Source> 

<SourceUrl>http://news.bbc.co.uk/go/rss/-/2/hi/middle_east/5398752.stm</SourceUrl> 
<DatePublished>Oct  2,  2006</DatePublished> 

<Summary>One  Palestinian  describes  how  the  Israeli  army  destroyed  his  house  with  half 
an  hour's  warning.</Summary> 

<RssFeedUrl>http://newsrss.bbc.co.uk/rss/newsonline_world_edition/middle_east/rss.xml 

</RssFeedUrl> 

<Type>News</Type> 

<Parents> 

<ParentId>Reuters_  1 24325235</ParentId> 

<ParentId>NPR_23 52 1 6 1 </ParentId> 

</Parents> 

<Key words  numWords="154"> 

cKeyword  frequency="6">israeli</Keyword> 
cKeyword  frequency="4">destroyed</Keyword> 
cKeyword  frequency="4">home</Keyword> 
cKeyword  frequency="4">housec/Keyword> 

c/Keywords> 

c/NewsArticle> 


Figure  2:  Metadata  of  a  JBI  information  object. 


A  JBI  client  is  any  software  application  (or  its  operator)  that 
makes  use  of  JBI  platform  services  to  publish,  subscribe, 
or  otherwise  interact  with  JBI  information  objects.  There 
are  two  kinds  of  clients  in  a  JBI-like  environment:  oper¬ 
ators  and  data  sources.  The  data  sources  such  as  sensors 
and  news  sources  can  publish  information  to  the  system, 
but  they  cannot  manipulate  information  objects  once  they 
are  published.  Instead,  an  operator  can  query,  manipulate 
and  merge  the  exchanged  information.  The  operator  can 
also  publish  the  merged  information  object  and  become  its 
owner  (the  publisher  of  the  object).  For  example,  in  the  bat¬ 
tlefield,  a  commander  may  publish  a  “spot  report”  based  on 
available  intelligence  reports,  sensor  data  and  other  back¬ 
ground  information  such  as  terrain  and  weather.  Simi¬ 
larly,  in  the  domain  of  intelligence  analysis,  an  analyst  often 
needs  to  assess  the  reports  based  on  his/her  understanding 
of  the  past  and  current  relevant  documents.  The  analyst  will 
also  write  reports  that  reflect  his/her  critical  thinking  and 
prediction  about  the  likely  course  of  action  of  a  specified 
event.  The  derived  information  published  in  the  system  can 
be  shared  with  other  analysts  with  similar  interests. 


2.2  Information  Pedigree 

In  order  to  allow  users  to  find  detailed  information,  a  de¬ 
rived  information  object  usually  contains  references  to  the 
objects  from  which  it  is  derived,  e.g.,  its  parents.  In  JBI 
this  “parents”  information  is  recorded  in  the  metadata  of  an 
information  object.  Given  the  parents  information  in  the 
metadata  of  information  objects,  we  can  easily  build  up  the 
pedigree  graph  for  the  derived  information  object.  Figure  3 
shows  an  example  of  pedigree  graph  for  a  derived  informa¬ 
tion  object  D.  As  shown  in  Figure  3,  if  the  user  merges 
three  pieces  of  information  A,  B,  and  C  to  create  a  fourth 
piece  of  information  D ,  the  parents  of  D  are  A,  B,  and  C. 
The  grandparents  of  D  are  the  union  of  the  parents  of  A,B, 
and  C  which  in  Figure  3  would  include  X,  Y,  and  Z. 


Figure  3:  An  example  of  the  pedigree  graph  for  information 
object  D. 

However,  for  information  derived  from  many  sources,  a 
brute  force  method  that  annotates  and  retrieves  entire  pedi¬ 
gree  information  with  a  tracking  log  may  be  combinatori- 
ally  expensive  in  storage  requirements  and  in  processing 
time.  We  have  developed  a  preliminary  framework  for  scal¬ 
able  pedigree  information  representation  and  exploitation, 
in  which  we  provide  a  suitable  representation  of  pedigree 
data,  information  chromosome  or  VChrome,  that  can  be  con¬ 
figured  to  provide  cost-apportioned  provenance  information 
given  an  operator’s  requirement. 

Specifically,  the  operators  can  explicitly  configure  the  scope 
of  provenance  in  the  pedigree  graph  within  certain  storage 
and  time  bounds.  For  example,  in  Figure  3,  an  operator 
might  only  want  to  access  the  upstream  nodes  of  2  hops 
away  from  object  D ,  e.g..  A,  B,  C,  X,  Y,  and  Z.  The 
motivation  here  is  that  the  size  of  the  annotations  may  grow 
exponentially  with  the  scope  of  a  pedigree  graph,  while  the 
utility  of  these  annotations  may  diminish  quickly  along  the 
path  from  the  derived  information  object. 

3.  Design  and  Implementation  of  AIMS 

In  this  section  we  first  describe  the  architecture  of  AIMS 
and  then  we  introduce  its  main  functionalities  such  as  query, 
publish,  and  subscribe. 


3.1  Architecture  of  AIMS 

The  AIMS  system  is  a  Java-based  lightweight  multiagent 
system  developed  for  managing  the  trustworthiness  of  in¬ 
formation  in  JBI-like  information  sharing  environments. 

In  AIMS  the  operator  obtains  information  from  the  infor¬ 
mation  sharing  environment  such  as  JBI  and  derives  new 
information  through  a  client  agent.  The  client  agent  pro¬ 
vides  the  primary  interface  to  facilitate  the  interactions  be¬ 
tween  a  client  and  the  JBI  repository.  For  example,  when  a 
client  publishes  an  information  object,  the  client  agent  will 
pass  the  derived  information  and  a  list  of  contributing  in¬ 
formation  to  the  Provenance  agent,  an  agent  for  retrieving 
pedigree  information  from  the  metadata  in  the  information 
object.  The  provenance  agent  establishes  either  complete  or 
approximate  pedigree  graph  to  calculate  various  qualities. 
The  mechanics  of  establishing  provenance  are  based  on  the 
system  defaults.  The  service  invocation  parameters  govern 
the  scope  of  provenance  to  be  established  by  some  subsets 
of  direct  and  indirect  contributing  information  objects  in  the 
pedigree  graph. 

3.2  Operations  in  AIMS 

The  AIMS  architecture  is  developed  for  JBI-like  publish- 
subscribe  information  sharing  environments.  Currently,  the 
AIMS  system  was  evaluated  by  a  specific  group  of  re¬ 
searchers  and  developers  at  Quantum  Leap  Innovations. 
The  system  automatically  extracts  news  articles  from  a 
number  of  RSS  (Really  Simple  Syndication)  news  feeds 
specified  by  a  user.  Examples  of  RSS  we  used  in  AIMS 
include  Yahoo  Iraq:  http://rss.news.yahoo.com/rss/iraq  and 
Yahoo  Middle  East:  http://rss.news.yahoo.com/rss/mideast. 
The  raw  XML-format  news  articles  are  parsed  and  stored 
into  an  information  repository,  which  is  implemented  using 
the  MySQL  database. 

There  are  two  kinds  of  operators  in  AIMS  system:  users 
and  superusers.  Users  of  such  an  environment  have  three 
main  operations  -  publish,  subscribe,  and  query.  Besides 
the  three  functionalities  a  normal  user  has,  a  superuser  also 
has  the  functions  of  managing  the  quality  library  and  user 
accounts.  For  example,  a  superuser  might  add  a  new  method 
to  calculate  the  trustworthiness  of  a  derived  information  ob¬ 
ject  besides  average ,  max  and  min,  depending  on  the  ap¬ 
plications  of  AIMS.  Also,  all  users  can  query  their  log  files 
about  their  past  operations. 

3.2.1  Register 

Sharing  a  broad  range  of  information  over  a  diverse  environ¬ 
ment  like  JBI  is  non-trivial  and  requires  careful  considera¬ 
tion  of  security  issues  such  as  validation  and  authentication 
of  information  sources.  For  example,  both  the  subscriber 
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Users  Sources 

Figure  4:  JBI  query,  publish  and  subscribe  interface. 


and  the  publisher  have  to  be  authenticated  by  the  JBI  plat¬ 
form  before  any  action.  At  the  same  time,  the  subscriber 
wants  to  make  sure  that  the  subscribed  information  object 
is  coming  from  a  legitimate  publisher. 


Figure  5:  JBI  logon  interface. 

The  current  version  of  AIMS  provides  a  simple  user  authen¬ 
tication  service  using  passwords.  When  a  user  first  comes 
up  to  the  AIMS  system,  he  needs  to  register  and  set  up  his 
password.  The  first  step  for  an  AIMS  user  is  to  log  on  his 
name  and  input  the  password  (see  Figure  5).  If  the  pass¬ 
word  is  correct,  AIMS  will  initialize  the  environment^. g., 
connect  to  server  socket  http  :  / / aims .quantumleap.us  : 
9080  and  the  corresponding  databases  on  the  server)  and  go 
to  the  main  window  of  AIMS.  Also,  a  user  in  AIMS  sys¬ 
tem  may  unregister  himself  from  JBI  platform  by  clicking 
on  the  “delete  user”  button  in  the  logon  window. 


3.2.2  Query,  Publish  and  Subscribe 

Figure  4  shows  the  main  window  of  AIMS,  which  is  imple¬ 
mented  using  Standard  Widget  Toolkit  (SWT)  -  a  graphical 
widget  toolkit  for  the  Java  platform  originally  developed  by 
IBM  and  maintained  now  by  the  Eclipse  Foundation. 

An  authenticated  user  can  type  any  text  (e.g.,  keywords)  to 
query  the  content  and  authors  of  information  objects  using 
SOAP/JDBC  protocols.  The  AIMS  system  will  pass  the 
query  as  an  XPath  expression  to  the  JBI  repository  and  re¬ 
turns  a  list  of  news  articles.  Note  that  actually  the  Xpath 
expression  only  searches  over  the  metadata  of  information 
objects.  The  user  can  click  on  the  title  of  an  article  to  view 
its  content,  parent  documents  and  quality  information.  The 
user  can  simultaneously  open  multiple  article  in  the  same 
window  and  each  tab  (in  the  bottom  part  of  the  window) 
corresponds  to  an  opened  article. 

The  AIMS  system  also  allows  the  user  to  summarize  the 
articles  in  the  repository  for  a  given  topic,  e.g.,  “Iraq”,  and 
derive  some  new  articles.  Figure  4  displays  the  edit  window 
for  a  derived  article  “Sample  Summary  Document”,  where 
the  user  can  either  type,  copy  and  paste  part  of  an  article. 
The  user  can  view  the  title,  author,  date  and  content  of  the 
derived  article  as  well  as  its  quality  and  pedigree.  When  the 
user  decides  to  publish  an  article,  the  AIMS  system  will  pop 
up  a  window  to  show  the  estimate  trust  for  the  derived  arti¬ 
cle.  The  user  may  then  choose  to  change  the  value  of  trust. 
The  changed  value  of  trust  is  used  to  learn  the  trustworthi¬ 
ness  of  parental  documents  and  reliabilities  of  clients.  Sec¬ 
tion  4  gives  more  details  of  our  learning  algorithm  in  AIMS. 


Moreover,  the  AIMS  system  allows  the  user  to  subscribe  to 
articles  published  in  the  repository.  This  is  reminiscent  of 
the  force  templates  used  by  commanders  of  different  eche¬ 
lons  in  the  battlefield,  but  here  we  only  support  some  simple 
features  of  subscribe  in  JBI,  where  a  user  can  only  specify 
the  content,  author  and  a  threshold  of  trust  for  subscribed 
news  articles. 

3.2.3  User  Management 


Figure  6:  JBI  monitoring  interface  for  superusers. 

Besides  the  basic  features  of  JBI,  such  as  query,  publish, 
and  subscribe,  AIMS  also  provides  a  tool  of  user  manage¬ 
ment  for  superusers.  Figure  6  shows  a  list  of  AIMS  users 
and  their  values  of  trust  for  both  operators  (users)  and  data 
sources.  The  tool  is  beneficial  to  a  community  of  intel¬ 
ligence  analysts,  as  the  supemser  can  easily  identify  and 
strength  and  weakness  of  each  analyst  in  a  given  task. 

4.  LEARNING  IN  AIMS 

One  contribution  of  AIMS  is  that  we  provide  a  learning 
mechanism  that  can  assess  the  changing  trustworthiness  of 
information  from  various  data  sources  in  dynamic  environ¬ 
ments.  In  AIMS,  a  user  can  change  the  value  of  trust  for 
a  derived  document.  The  value  is  captured  as  feedback 
and  integrated  into  the  AIMS  system  to  better  assess  the 
trustworthiness  of  documents  in  the  future.  In  this  section 
we  describe  the  details  of  our  learning  algorithm  in  AIMS. 
Specifically,  we  extend  the  existing  JBI  model  in  two  ways, 

•  Trustworthiness  of  an  object  A  scalar  t  is  used  to 
model  the  degree  of  trust  on  an  information  object, 
where  0  <  t  <  1. 

•  Reliability  of  a  client  The  reliability  of  each  client  Sj 
is  captured  as  a  scalar  r,  in  [0, 1],  which  is  dynamically 
adjusted  based  on  the  feedback  from  operators  for  de¬ 
rived  information  objects.  The  value  of  r,  is  close  to 
zero  if  the  entity  or  source  is  unreliable. 


The  trustworthiness  of  each  information  object  is  stored  as 
an  additional  entry  in  the  metadata  of  the  object.  We  also 
introduce  a  database  in  JBI  to  store  the  reliability  of  all  reg¬ 
istered  clients,  including  both  operators  and  data  sources. 

In  order  to  facilitate  the  interactions  between  operators  and 
the  system,  each  operator  is  associated  with  a  software 
agent,  called  client  agent.  Moreover,  there  is  a  provenance 
agent  in  the  system.  The  provenance  agent  will  assess  the 
metadata  of  each  information  object  and  reliabilities  of  each 
client.  The  provenance  agent  will  provide  the  following  two 
services  to  all  client  agents, 

•  Reason  about  the  trustworthiness  of  a  derived  informa¬ 
tion  object. 

•  Retrieve,  track  and  update  the  reliabilities  of  clients. 

The  client  agent  collaborates  with  the  provenance  agent  to 
annotate  the  calculated  trustworthiness  of  a  derived  infor¬ 
mation  object  in  its  metadata  when  the  client  .st  publishes 
the  object  into  the  repository. 

4.1  Trustworthiness  of  Derived  Objects 

Obviously,  information  from  reliable  clients  usually  has 
higher  quality  than  the  one  provided  by  unreliable  sources. 
In  this  paper  we  use  a  scalar  r*  to  represent  the  reliability 
of  a  client  Sj.  Given  each  client’s  belief  in  the  informa¬ 
tion  object  and  the  client’s  reliability,  the  agent’s  belief  in 
the  derived  object  can  be  computed  in  many  different  ways. 
For  simplicity,  here  we  only  consider  the  effects  of  its  direct 
parents  in  the  pedigree  graph  on  its  trustworthiness. 

The  initial  trustworthiness  of  an  information  object  is  as¬ 
signed  to  0.5.  Suppose  d  is  derived  from  a  list  of  informa¬ 
tion  objects  {d\ ,  d2,  ■  ■  ■ ,  di,}  by  client  s  and  the  provenance 
agent  will  estimate  the  trustworthiness  of  object  d.  The 
trustworthiness  of  object  d  can  be  determined  as  the  average 
of  trust  worthiness  of  corresponding  objects  di,  d%, . . . ,  dL, 

yL_  u 

e.g.,  — ,  where  tt  is  the  trustworthiness  of  document 

di. 

4.2  Learning  the  Reliability  of  Clients 

The  value  of  trust  can  be  adjusted  by  an  operator  if  he/she 
finds  the  estimated  trustworthiness  of  the  object  by  the  agent 
is  not  correct.  1  For  example,  the  client  can  simply  change 
the  degree  of  trust  based  on  his/her  experiences.  Formally 
we  can  define  positive  and  negative  feedback  for  an  infor¬ 
mation  object  as  follows. 

Definition  1  Suppose  t  is  the  trustworthiness  for  a  derived 
object  d,  and  t'  is  the  adjusted  value  of  trust  by  the  client,  t' 

1  Note  that  in  JBI  an  operator  can  change  the  value  of  trust  for  an  object 
only  when  he/she  publishes  it. 


is  considered  to  be  a  positive  feedback  if  and  only  if  t  <  t'\ 
otherwise,  t!  is  a  negative  one. 

Given  the  estimated  trust  t  and  the  feedback  t'  for  a  derived 
object  d,  the  trustworthiness  of  object  d  will  be  adjusted  as 

f. 

The  next  question  is  how  to  utilize  the  feedback  to  update 
the  reliability  r,  for  a  client  Sj,  where  d,  is  one  of  the  parents 
of  d  and  di  is  published  by  client  st.  The  basic  idea  here  is  to 
estimate  the  distance  between  t  and  t' .  A  client  will  receive 
more  penalty  for  its  reliability  if  there  is  a  larger  distance. 
Now  let’s  study  the  mechanism  of  updating  the  reliability 
of  other  clients  corresponding  to  the  information  objects  in 
the  pedigree  graph. 

For  an  information  object  d  that  is  derived  from  a  list  of  ob¬ 
jects  {d±,  di,  ■  ■  • ,  di}  by  client  s,  the  owner  of  d  is  denoted 
as  owner(d)  =  s  and  the  direct  parents  of  d  are  denoted 
as  parents(d)  =  {di,  d2,  ■  ■  ■ ,  d l}.  From  the  metadata  of 
each  parent  di  £  parents(d),  we  can  easily  build  up  the 
pedigree  graph  for  information  object  d.  Note  that  a  client 
can  be  the  publisher  of  multiple  information  objects  in  a 
pedigree  graph. 

Definition  2  A  pedigree  graph  for  a  derived  information 
object  d  can  be  defined  as  a  directed  graph  G  =  {d,  A,  H }, 
where  A  is  the  set  of  derivative  information  objects  and  H 
is  the  maximal  hop  number  from  an  information  object  to 

d. 

Note  that  the  value  of  the  reliability  of  a  client  rt  is  bounded 
by  0  and  1.  Therefore,  when  we  update  the  reliability  for 
each  client,  we  cannot  increase  or  decrease  it  infinitely. 
Next  we  give  one  definition  to  capture  the  intuition.  Its 
value  increases  gradually  to  one  given  the  positive  feed¬ 
back,  but  decreases  linearly  for  negative  feedback.  The  idea 
here  is  that  the  reliability  should  be  hard  to  build  up,  but 
easy  to  tear  down.  The  following  definition  gives  a  rule  to 
update  the  reliability  of  the  publisher  for  an  information  ob¬ 
ject. 

Definition  3  Given  the  distance  p  between  t  and  t'  for  an 
object  d  and  p  =  \t— 1'\,  for  any  of  its  parents  .sy,  its  reliabil¬ 
ity  can  be  calculated  as  r\  =11  (ri,  p)  =  ri  +  p(l  —  rf)  if  the 
feedback  is  positive;  otherwise,  r\  =  II(ri,  p)  =  ri(  1  —  p). 

Besides  the  reliability  of  the  publisher  of  object  d,  the  client 
agent  needs  to  update  the  reliability  of  other  clients  whose 
objects  are  used  to  derive  new  information  objects.  Algo¬ 
rithm  1  summarizes  feedback  propagation  algorithm,  where 
we  use  a  breadth-first  search  to  propagate  credits/penalties. 
One  key  issue  here  is  to  deal  with  cycles  in  the  pedigree 
graph,  e.g.,  the  cycle  C  — >Y  — >  Z  — >  C  in  Figure  3.  In  our 
algorithm,  the  agent  stores  the  visited  information  objects 
into  a  list  updateP arents  during  the  propagation  process. 


For  a  given  information  object  in  the  pedigree  graph,  we 
only  update  its  client  once  regarding  to  this  object.  Note 
that,  a  client  can  be  updated  multiple  times  for  a  given  pedi¬ 
gree  graph.  Moreover,  the  client  agent  of  s  also  maintains 
a  list  of  information  objects  at  the  current  hop  currentLevel 
and  a  list  of  information  objects  to  be  visited  at  the  next  hop 
nextLevel. 


Algorithm  1  Algorithm  p ro p ag at e Fe cd  b ac  l< (VI ,  p.  H) 

1:  Initially,  we  have  hop  number  h  =  0, 

updatedP arents  =  {},  currentLevel  =  {}, 
and  nextLevel  =  {}. 

2:  Update  the  reliability  of  s,  where  s  =  owner  (d) 

3:  updatedP arents  =  {d} 

4:  currentLevel  =  {d} 

5:  while  h  <  H  do 
6:  while  currentLevel  {}  do 

7:  For  each  di  £  currentLevel 

8:  Si  =  owner  (df) 

9:  nextLevel  =  nextLevel  U  parents(di) 

10:  currentLevel  =  currentLevel  —  di 

11:  if  Sj  f  updatedP  arents  then 

12:  Update  the  reliability  for  s,;  by  II(rj,  p/2h). 

13:  updatedP  arents  =  updatedP arents  U  {di)} 

14:  end  if 

15:  end  while 

16:  h  =  h  +  1 

17:  For  each  di  £  nextLevel 

18:  currentLevel  =  currentLevel  U  {di} 

19:  nextLevel  =  nextLevel  —  {di} 

20:  end  while 


For  a  derived  object  in  its  pedigree  graph,  different  objects 
may  have  different  contributions  depending  on  their  dis¬ 
tance  to  the  derived  object  d.  Here  we  assume  the  distance 
from  object  d  to  object  di  is  h  hops,  where  h  =  0  for  object 
d.  Moreover,  we  can  specify  the  scope  of  propagation  in  the 
algorithm,  e.g.,  H  =  2  hops.  The  value  of  H  can  be  eas¬ 
ily  configured  in  the  AIMS  system  (see  Figure  4).  In  each 
level  of  propagation,  a  client  in  the  pedigree  graph  will  get  a 
credit/penalty  of  p/2h  from  the  derived  information  object. 

Figure  7  shows  a  simple  example  of  learning  in  AIMS, 
where  document  D  is  derived  from  documents  A,  B,  and 
C.  Assume  the  value  of  estimated  trust  for  document  D 
is  0.5  and  the  feedback  from  the  user  4  is  0.8.  The  AIMS 
system  will  update  the  trustworthiness  of  document  D  as 
0.8  and  the  reliabilities  of  clients  1,  2,  3,  4  as  0.575,  0.575, 
0.575,  0.65,  respectively,  assuming  the  initial  values  of  re¬ 
liability  for  all  of  them  are  0.5.  Figure  8  shows  the  changes 
of  reliabilities  for  both  operators  (users)  and  data  sources  in 
AIMS. 


Figure  7:  An  example  of  learning  in  AIMS,  where  the  cir¬ 
cles  denote  clients  and  rectangles  denote  the  information 
objects  . 


5.  RELATED  WORK 


Recent  research  in  trust  is  motivated  by  large-scale  open 
and  distributed  applications  such  as  resource  sharing  and 
electronic  commerce. 

One  common  way  of  assessing  tmst  are  through  using  rep¬ 
utation  mechanisms.  Researchers  have  developed  various 
distributed  reputation  mechanisms  for  ratings  representa¬ 
tion,  collection,  and  aggregation  in  peer-to-peer  systems 
and  multiagent  systems  [4,  10,  18,  19,  20].  More  recently, 
[17]  discusses  trust  representation  and  aggregation  via  a 
probability  certainty  distribution.  However,  most  of  them 
cannot  be  directly  used  for  tracking  and  calculating  vary¬ 
ing  quality  of  information  published  by  different  clients  in 
JBI-like  environments. 

In  the  Semantic  Web,  [14]  gives  an  approach  to  trust  man¬ 
agement  based  on  path  algebra.  He  assumes  there  are  to¬ 
tal  M  users  and  N  statements  and  defines  a  class  of  func¬ 
tions  for  merging  the  trust  matrix  from  different  users  using 
path  algebra.  In  our  system  both  M  and  N  can  be  dynam¬ 
ically  changing.  Moreover,  learning  is  not  considered  in 
[14],  [13]’s  approach  is  similar  to  [14],  but  the  difference  is 
that  EigenTrust  computes  a  global  trust  value,  instead  of  a 
personalized  view  of  trust. 

Some  researchers  study  a  similar  problem  in  web-based  in¬ 
formation  systems,  but  none  of  them  consider  the  use  of 
pedigree  information  for  trust  assessment.  For  example, 
Gil  and  Ratnaker  develop  a  system  to  derive  consensus  trust 
on  sources  in  a  community  of  users  as  they  use  or  dismiss 
sources  for  information  analysis  tasks  [6].  Also,  [2]  exam¬ 
ines  the  cognitive  side  of  trust  in  information  sources.  [2] 
proposes  a  model  for  making  trust  decisions  about  sources, 
differentiating  internal  and  external  attributes  affecting  trust 
in  a  source.  The  authors  note  that  the  composition  of  the  in¬ 
puts  to  a  trust  decision  affect  the  decision,  and  thus  the  de¬ 
cision  itself  can  not  be  characterized  by  a  final  probability. 


Figure  8:  Screenshots  of  the  changes  of  reliabilities  for  both 
operators  (users)  and  data  sources. 


6.  CONCLUSION 

In  this  paper  we  present  AIMS,  a  prototype  implementation 
of  a  JBI  publish-subscribe  information  sharing  system.  The 
prototype  system  provides  a  basis  for  the  development  and 
evaluation  of  mechanisms  of  managing  the  quality  of  infor¬ 
mation  in  dynamic  information  sharing  environments.  The 
examples  show  that  AIMS  can  capture,  retrieve,  and  reason 
about  the  trustworthiness  of  information  and  reliabilities  of 
clients  in  an  effective  and  coherent  way. 

In  this  paper  we  only  consider  the  upstream  learning  from 
the  users’  feedback,  where  feedback  is  propagated  from  a 
derived  object  to  its  parents  and  grandparents.  In  practice, 
the  change  of  source  information  may  also  affect  the  trust¬ 
worthiness  of  derived  information.  In  the  future  work  we 
plan  to  develop  similar  techniques  for  downstream  learning, 
where  the  agents  can  automatically  propagate  the  changes 
of  source  information  and  update  all  derived  information 
objects. 

Moreover,  we  plan  to  study  the  synergies  between  symbolic 
arguments  and  probabilistic  reasoning  [5,  7].  The  motiva¬ 
tion  is  that  two  users  might  get  different  conclusions  even 
they  choose  a  similar  set  of  documents  for  analysis.  The 
basic  idea  is  that  we  can  trace  the  pedigree  graph  and  com¬ 
pare  the  strength  of  their  conclusions  using  some  probabilis¬ 
tic  argumentation  techniques.  The  method  will  enable  the 
newbie  with  less  experience  to  share  the  expertise  from  ex¬ 
perienced  analysts  in  the  collaborative  information  sharing 
systems. 
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